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ABSTRACT 

We developed a machine vision system to automatically 
capture the dynamics of pedestrians under four different 
traffic scenarios. By considering the overhead view of each 
pedestrian as a digital object, the system processes the 
image sequences to track the pedestrians. Considering the 
perspective effect of the camera lens and the projected 
area of the hallway at the top-view scene, the distance of 
each tracked object from its original position to its cur¬ 
rent position is approximated every video frame. Using 
the approximated distance and the video frame rate (30 
frames per second), the respective velocity and acceleration 
of each tracked object are later derived. The quantified 
motion characteristics of the pedestrians are displayed by 
the system through 2-dimensional graphs of the kinematics 
of motion. The system also outputs video images of the 
pedestrians with superimposed markers for tracking. These 
visual markers were used to visually describe and quan¬ 
tify the behavior of the pedestrians under different traffic 
scenarios. 

Keywords 

machine vision, crowd dynamics, pedestrian traffic, multiple 
tracking, color tracking, motion 

1. INTRODUCTION 

Pedestrian traffic is the behavioral flow of people in an area 
or moving crowd. In any publicly-accessible area, varia¬ 
tions of pedestrian traffic can always be observed. There 
are several places in these public areas where the sudden 
change in the behavior of huge groups of people can cause 
traffic jams, specifically along hallways and on crossroads. 
The measurement of the dynamics of pedestrian traffic is 


important in various applications and field of studies such as 
crowd control, stampede and accident control, public secu¬ 
rity, pedestrian flow organization, pedestrian safety, archi¬ 
tectural design and safety, advertising, and marketing. For 
example, architects and transportation designers frequently 
use crowd dynamics in designing environments and buildings 
for the efficiency of pedestrian flow. During extreme condi¬ 
tions, such as stampedes, riots, and fire escapes, improved 
procedures on building designs could save many lives [la¬ 
in obtaining data about the dynamics of pedestrian traffic, 
the usual procedure is to manually observe the various 
metrics associated with particle motion of a pedestrian by 
assigning several researchers on top of walkways over a 
period of time. This manual procedure is tedious, prone 
to error, and sometimes unreliable because of the human 
factors affecting the observers. During the start of the 
observation, the data gathered might be accurate because 
the human observers are still attentive. However, towards 
the later part of the observation, the data may become 
inaccurate due to the observers’ boredom, eye stress, and 
tiredness. Thus, to replace the human observers, we created 
a computer vision system to mimic the observing capabili¬ 
ties of humans at a higher accuracy and consistency during 
the longest possible time. This study aims to effectively 
present the pedestrian behavior and characterize the motion 
of the crowd using the developed vision-based system. 

Several researchers have attempted to simulate the pedes¬ 
trian traffic dynamics wherein models for crowd dynamics 
were developed [TT]. Some of these models are the agent- 
based model of Reynolds [12] , the social force model of Hel- 
bing and Molnar [5] , the cellular automata model of Blue [5] , 
and the gas-kinetic model of Helbing |^, but all of them 
used data gathered by human observers. To develop better 
models, data must be gathered by a system that is less prone 
to error due to human subjectivity and tiredness, and thus, 
a vision-based system needs to be developed. To develop our 
vision-based system, we utilized models, concepts, and tech¬ 
niques in image processing and computer vision. Our aim 
for the system is to detect and track pedestrians and at the 
same time measure and analyze the pedestrian’s collective 
behavior under four different traffic scenarios. The system 


accepts as input the overhead view of the pedestrian traffic. 
To aid the system operator in real-time, color is used to 
track multiple pedestrians. The system, then, outputs sev¬ 
eral metrics and captures the dynamics of these metrics in 
2-dimensional graphs of the kinematics of motion: distance 
vs. time, velocity vs. time, and acceleration vs. time. 

2. RELATED LITERATURE 

2.1 Recent Studies in Traffic Dynamics 

Because of the importance of crowd dynamics on several 
real-world applications, several researchers attempted to 
quantify the collective dynamics of the pedestrians through 
developed simulations of motion. Helbing and Molnar [9], 
borrowing some ideas in gas-kinematic models, introduced 
the social force model to simulate pedestrian flows. In 
their model, a self-driven particle (i.e., a pedestrian) that 
interacts through social rules and regulations tries to move 
in its desired speed and direction while at the same time 
attempts not to collide with obstacles, other particles, and 
surrounding barriers. In order to reach its destination faster, 
pedestrians take detours even if the route is crowded |10 |. 
The choice, however, is dependent on the recent memory of 
what the traffic was like the last time they took the route, 
which was found by other researcher to be polygonal in 
nature [4]. In agreement with the social force model, Weid- 
mann m observed that, as long as it is not necessary to go 
faster, such as going down a ramp, a pedestrian prefers to 
walk with his or her desired speed, which corresponds to the 
most comfortable walking speed. However, Weidmann fur¬ 
ther observed that pedestrians keep a certain distance from 
other pedestrians and borders. The distance between the 
pedestrians decreases as the density of the crowd increases. 
The pedestrians themselves cause delays and obstructions. 
Arns [2] observed that the motion of the crowd is similar 
to the motion of gases and fluid, while Helbing, et al. | 10 | 
suggested that it is similar to granular flow as well. 

Helbing [7], in his extension of the social force model, showed 
that many aspects of traffic flow can be reflected by self- 
driven many-particle systems. In this system, he identified 
the various factors that govern the dynamics of the parti¬ 
cles such as the specification of the desired velocities and 
directions of motion, the geometry of the boundary pro¬ 
files, the heterogeneity among the particles, and the degree 
of fluctuations. One such observable pattern is the forma¬ 
tion of lanes of uniform walking direction, formed because of 
the self-organization of the pedestrians [T]. Aside from the 
self-organizing behavior of the crowd, obstacles were also 
observed to both positively and negatively contribute to the 
flow of the traffic. 

During escape panic of large crowds, several behavioral phe¬ 
nomena were observed [S]: build up of pressure, clogging 
effects at bottlenecks, jamming at room widening areas, 
faster-is-slower ejfect, inefficient use of alternative exits, ini¬ 
tiation of panics by counter flows, and impatience. It was 
observed that the main contributing behaviors in these sit¬ 
uations is a mixture of individual and grouping behavior. 


2.2 Tracking and Detecting Objects 

Detection and tracking of objects using machine vision is 
a much efficient solution, with much higher consistency 
of results, than the canonical manual observation per¬ 
formed by multiple human observers. Several methods have 
been implemented and tested in the area of machine-based 
tracking, the most traditional of which are the background 
subtraction method and the frame differencing method. 
However, the most efficient approach m by far is by uti¬ 
lizing a bipartite graph . In this method, contains 
classes that represent certain entities about a frame or image 
parsed from a video. One class corresponds to the expected 
positions of the tracked objects Ef £ while the other 

corresponds to the blobs B from within the current image. 
Features such as size, position, color, shape, and velocity 
may be extracted from the object blobs. An adjacency 
matrix Mq of the graph will be created, wherein the matrix 
entries niij are the similarity measurements between the 
jth blob and ith tracked object. 

2.3 Motion Analysis 

Researches abound in the study of motion in image 
sequences. Motion analysis is the science of comparing 
sequential still images captured from photographing a body 
in motion in order to study the kinematies (the motion 
themselves) and the kinetics (external and internal forces) 
of the body [5]. Often called dynamic image analysis | 14 |. 
motion analysis is only based on a few number of consecu¬ 
tive frames, which has similar methods as to the analysis of 
static images. Correspondence between the pairs of points 
of interest are being observed in the sequence of images. It 
uses the concept of motion held which projects the three- 
dimensional motion into two-dimensional one, such that a 
velocity vector (e.g., direction, velocity, and distance) from 
an observer is assigned to each point in the two-dimensional 
image. A good approximation of the motion field is the 
concept of the optical flow. Optical flow gives a description 
of motion and can be a valuable contribution to the inter¬ 
pretation of the image. Since the data is qualitative, optical 
flow can be done even if no quantitative data are obtained 
from motion analysis. Furthermore, optical flow can be used 
to study a large variety of motions, as it can determine the 
motion direction and velocity at image points. The method, 
however, suffers in accuracy due to illumination changes 
but can still be useful by taking in some motion-localized 
assumptions, such as constraining the motion to maximum 
velocity, small acceleration, common motion, and mutual 
correspondence. 

3. METHOD 
3.1 Video Capturing 

We captured several videos in overhead view of pedestrians 
along hallways and cross ways under different traffic sce¬ 
narios as follows: 

1. Scenario 1: Pedestrians were flowing smoothly in one 
direction (left-to-right, LR). 


Input Video 



I OutputVideo 


Figure 1: Flowchart of the system showing major 
processes. 


2. Scenario 2: Pedestrians were flowing in two opposite 
directions simultaneously (LR and right-to-left, RL). 
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Figure 2: Study area (a) corridor, (b) lobby, (c) 
overhead view of the corridor, and (d) overhead view 
of the lobby. This figure is in color in the electronic 
copy of this paper. 


3. Scenario 3: Pedestrians were initially flowing from a 
single direction (LR) and then suddenly, a pedestrian 
stops in the middle and makes a U-turn. 

4. Scenario 4: Pedestrians were moving against counter 
flows: They were passing by each other in several and 
inconsistent pathways or lanes. 

In the corridor of the study area (Figure [2] (a)), after consid¬ 
ering for the perspective effect of the camera lens, the dimen¬ 
sions of the captured area (Figure [2] (c)) is 2.0 x 1.5 (3 m^ ) 
when the camera was mounted 3.5 m from the hallway floor 
(Figure[2] (b)). However, the captured area (Figure[2](d)) is 
4.5 m^ (2.5 X 1.8) when the camera is mounted 3.9 m from 
the ground. We used a video rate of 30 frames per second 
(fps). 

3.2 Preprocessing Image Sequences 

We split into several frames F — {fi, f 2 ,■ fn} the respec¬ 
tive n-frame videos of the different pedestrian scenarios 
using a video processing software called VirtualDub (Ver¬ 
sion 1.6.16) [ 15 ]. Because of the variability of the scenarios, 
each has a different value for n. We then archived the image 
frames in file directories. 


3.3 Detecting Pedestrians 

We used the hair color of the pedestrians to detect them by 
utilizing various intensities of the black pixels. We did this 
by scanning each /i,Vi = 1,2, ...,n through a bounding 
box b, and then by processing the two-dimensional pixels 
bounded by b. Within each b, we counted the valid red- 
green-blue (RGB) color values of black pixels. We indicated 
that b has already bounded a candidate pedestrian head 
once a threshold sum of pixels is met (See Figure [3] for 
an example overhead image with detected head). We then 
searched the vicinity margin of b for more related pixels. 
Once done, we incremented the position of b and repeated 
the same process for the next set of pixels bounded by b. 

3.4 Tracking Pedestrians 

Once we have identified that b bounds a pedestrian’s head, 
we computed the two-dimensional coordinates of the center 
of mass Mx,y{0) of the bounded object O. We used Mx,y 
to track the coordinates of the center of mass of the pedes¬ 
trian’s head. We considered b to be still bounding the pedes¬ 
trian’s head if the difference of the current (xt+i,yt+i) and 
previous {xt, yt) of M{0) is less than a set value (See Table[T] 
for the set of values we used in this study). Otherwise, we 
considered the / as erroneous and used Mx,y{0) at time t 














F« -1 


Figure 3: An image frame showing an example of a 
pedestrian’s head as detected by the procedure and 
bounded by b. This figure is in color in the electronic 
copy of this paper. 


instead. Because the pedestrians head might overlap in real- 
world due to the perspective effect of the camera lens, we 
allowed b to overlap with other bs in a certain margin (See 
column F of Table [T] for the margin values we used in this 
study). We repeated the process for the succeeding / until 
the object being tracked is out of the scene. We consid¬ 
ered an object as out of the scene when there is no more 
valid number of black pixels within b that can represent the 
object. 

We utilized several threshold values to account for the varied 
lighting conditions when the input image sequences were 
recorded for each different scenario. We did this to avoid 
false detection and misdetection of pedestrians in the image 
sequences. The values of variables we used for the different 
scenarios are shown in Table [T] 


Table 1: Different values of variables for different 
scenarios. A - the dimensions of 6; B - Maximum 
RGB color value of black pixel for the object; C - 
Minimum number of valid black pixels; D - Vicinity 
margin of the object (in number of pixels); E - Max¬ 
imum difference of the current and previous Mx,y{0) 
(in number of pixels); F - Margin for overlapping 
with other objects’ bounding boxes (in number of 
pixels). 


Scenario 


A 

B 

C 

D 

E 

F 

Scenario 1 

50 

X 50 pixels 

25 

300 

0 

17 

7 

Scenario 2 

50 

X 50 pixels 

22 

450 

6 

30 

20 

Scenario 3 

40 

X 40 pixels 

30 

600 

0 

30 

20 

Scenario 4 

40 

X 40 pixels 

30 

450 

0 

30 

20 


To track multiple objects, we colored the outline of b as 
white once b detects an object. This way, the system will 
not keep detecting and tracking the same object within the 
same /. 


3.5 Data Computation 

3.5.1 Tracing Trajectories and Computing Distances 
We traced the trajectories or the path traveled of each pedes¬ 
trian in the image using each object’s Mx^y(0) for every /. 
Given the coordinates of the [x, y) in each /, we used the 
cononical two-dimensional distance formula, the L 2 metric, 
to compute for the distance dp{0) traveled by each pedes¬ 
trian per /. 

3.5.2 Computing Velocities and Accelerations 
Given the change in the position {Sdp{0)) over the change 
in time (St) of each tracked head, we obtained the instan¬ 
taneous velocity Vp{0) at each / of each tracked pedestrian 
by a simple ratio Sdp{0)/Stp. We computed the acceler¬ 
ation ap{0) of each tracked pedestrian by the change in 
velocity over time {5vp{0)/5t). 

3.6 Presentation of Measured Values 

We presented the trajectories of the pedestrians by superim¬ 
posing colored lines following the center of mass M^.y on the 
video scene (Figured]). We displayed the other motion char¬ 
acteristics of pedestrians as line graphs over time: distance 
vs. time, velocity vs. time, and acceleration vs. time. 

4. RESULTS AND DISCUSSION 

In comparing the motion characteristics of the pedestrians at 
different scenarios, we used the trajectories (superimposed 
on video sequences) and the graphs generated by the system. 

4.1 Scenario 1 

This scenario shows a freely flowing pedestrians. As cap¬ 
tured in the output graphs (Figure [6|), uniform speed is 
shown for all the pedestrians. The small variation shown in 
the graph can be attributed to the integer arithmetic when 
dealing with pixel positions and not with the small motion 
by the pedestrians. However, even if such motion was cre¬ 
ated by the pedestrians, such variations are small enough to 
create traffic jams. 

4.2 Scenario 2 

In this scenario, the traffic fiow of simultaneous crossing of 
pedestrians from opposite directions is smooth, as captured 
by the velocity vs. time graph in Figure [7| Although there 
were two directions of pedestrian flow, the individuals were 
moving in uniform lanes. We observed in their respective 
trajectories (Figure [7] (b)) that there were two dominant 
divisions of walking lanes. This observation surely verifies 
the claim of Helbing [1] that the pedestrians self-organize. 
Figure [7](c) shows the separate groups of pedestrians. The 
line graphs starting in distance zero correspond to the pedes¬ 
trians coming from the left part of the scenario while the 
other line graphs correspond to the ones coming from the 
right. Even if the graphs crossed each other, there were no 
major changes in the magnitude of the pedestrians’ motion 
characteristics. The pedestrians still created uniform lanes 
of walking direction to avoid moving delays. 
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Figure 4: An example output frame of the system. The upper Left part contains the input image with 
trackers (bounding boxes). The upper Right part contains the trajectory of the pedestrians. The lower part 
contains the distance vs. time graph of the trajectory. This figure is in color in the electronic copy of this 
paper. 


4.3 Scenario 3 

In this scenario, a pedestrian P stopped in an area in the 
scene and went back to the direction he came from (i.e., 
made a U-turn). It was observed in the graphs (Figure [8| 
that some pedestrians slowed down when they where inter¬ 
sected by P. Others stopped for a certain range of time. 
As P moves away from the other pedestrians, the latter 
slowly recovered and increased their velocity. It shows here 
that a pedestrian itself can cause delays in pedestrian traffic. 

4.4 Scenario 4 

In this scenario, a counter flow exhibited slow down of 
motion of the pedestrians before and during the intersec¬ 
tion. We observed no self-organization among pedestrians 
here. Instead, the individuals moved to different directions 
just to reach their destination and to avoid colliding with 
other pedestrians. We observed from some of the pedes¬ 
trians’ trajectories that they make turning motion to avoid 
the obstacles along their way (see Figure [5}. Even with this 
obstacle avoidance behavior, because everybody is avoiding 


the counter flow, a short-term traffic jam that lasted for a 
short time was observed. When a pedestrian moved out of 
the intersection, the pedestrian was able to get back to a 
uniform speed, as evident in Figure [9] 

5. CONCLUSION 

In this study, we created a machine vision system that auto¬ 
matically captures the dynamics of pedestrian traffic. The 
system was able to detect and track multiple pedestrians 
and effectively display some of their motion characteristics. 
We used the system to analyze and quantify the behavior of 
the pedestrian in four different traffic scenarios, namely: 

1. Scenario 1: Pedestrians were flowing smoothly in one 
direction (left-to-right, LR). 

2. Scenario 2: Pedestrians were flowing in two opposite 
directions simultaneously (LR and right-to-left, RL). 







Figure 5: The encircled path shows the trajectory of 
a right turn made by a pedestrian to avoid colliding 
with the incoming traffic. This figure is in color in 
the electronic copy of this paper. 

3. Scenario 3: Pedestrians were initially flowing from a 
single direction (LR) and then suddenly, a pedestrian 
stops in the middle and makes a U-turn. 

4. Scenario 4: Pedestrians were moving against counter 
flows: They were passing by each other in several and 
inconsistent pathways or lanes. 

Using the measured metrics by the system, the results of 
our analysis suggest that pedestrians themselves alter the 
traffic condition. The motion characteristics of a pedestrian 
depend on its desired magnitude and on the delays in its 
environment. In counter flows (as in Scenario 4), if pedes¬ 
trians self-organize, the traffic flow can be smooth, otherwise 
a traffic jam may build up. 

By learning the behavior of pedestrians in different traffic 
scenarios, we could efficiently devise safety and security mea¬ 
sures in crowded areas, avoid accident, injuries, and even loss 
of lives. This system will be helpful in the field of architec¬ 
ture for the structural designs of pedestrian hallways and 
pathways in buildings. We recommend this system for use 
in the study of pedestrian evacuation during fire escaping 
and for avoidance accidents such as stampedes. 

6. ACKNOWLEDGMENTS 

We value the unselfish technical inputs of Dr. Vladimir Y. 
Mariano, Associate Professor of the Institute of Computer 
Science, UPLB, in the area of image processing and video 
analysis. We also thank the Institute of Computer Science, 
UPLB, for the use of its computing equipments to create, 
and test the system. Lastly, we thank the unnamed pedes¬ 
trians that we captured in our videos. Unknown to them, 
they had been willing experimental subjects in our study. 

7. REFERENCES 

[1] D. Helbing (2). Safety management at large events: 
The problem of crowd panic, 2006. 


[2] T. Arns. Video films of pedestrian crowds, 1993. 

[3] V. J. Blue. Cellular automata microsimulation for 
modeling bi-directional pedestrian walkways. 
Transportation Research Part B: Methodological, 
35(3):293-312, 2001. 

[4] J. Ganem. A behavioral demonstration of Fermat’s 
principle. The Physics Teacher, 36, 1998. 

[5] 1. Griffiths. Principles of Biomechanics and Motion 
Analysis. Lippincott Williams and Wilkins, USA, 

2006. 

[6] D. Helbing. A fluid dynamic model for the movement 
of pedestrians. Complex Systems, 6:391, 1992. 

[7] D. Helbing. Traffic and related self-driven 
many-particle systems. Reviews of Modern Physics, 
73:1067, 2001. 

[8] D. Helbing, 1. Farkas, and T. Vicsek. Simulating 
dynamical features of escape panic. Nature, 

407(6803):487-490, 2000. 

[9] D. Helbing and P. Molnar. Social force model for 
pedestrian dynamics. Physical Review E, 51(5):4282-|-, 
1995. 

[10] D. Helbing, P. Molnar, 1. Farkas, and K. Bolay. 
Self-organizing pedestrian movement. Environment 
and Planning B: Planning and Design, 28:361-383, 
2001 . 

[11] J. A. Kirkland and A. A. Maciejewski. A simulation 
attempts to influence crowd dynamics. In SMC ’03 
Conference proceedings : 2003 IEEE International 
Conference on Systems Man and Cybernetics, pages 
4328-4333, Washington, D.C., 2003. 

[12] C. W. Reynolds. Flocks, herds, and schools: A 
distributed behavioral model. Computer Craphics, 
21(4):25-34, 1987. 

[13] M. Rowan and F. Maire. An efficient multiple object 
vision tracking system using bipartite graph matching. 
Technical report. School of Software Engineering and 
Data Communication, Queensland University of 
Technology, Australia, 2006. 

[14] M. Sonka, V. Hlavac, and R. Boyle. Image processing, 
analysis, and machine vision third edition, 2008. 

[15] virtualdub.org. VirtualDub version 1.6.16, 2009. 
http://www.virtualdub.org. 

[16] U. Weidmann. Transportation technique for 
pedestrians, 1993. 

About the Authors 

Louie Vincent A. Ngoho is an undergraduate student at 
UPLB. He is a member of the Red Cross Youth of UPLB 
and of the UPLB DOST Scholars’ Society. 

Jaderick P. Pabico is an Associate Professor of the Insti¬ 
tute of Computer Science, UPLB. He is one the 2008 Ten 
Outstanding Young Scientists of the Philippines. His 
research interests are in the areas of high-performance com¬ 
puting, intelligent computing, complex systems, computer 
security and forensics, simulation and modeling, and game 
programming. 








TH* !■ 


(d> 





Figure 6: (a) An output image, (b) the corresponding trajectories of the pedestrians, (c) the distance vs time 
graph, (d) the velocity vs time graph, and (e) the acceleration vs time graph for the pedestrians in Scenario 
1. Each colored set of points in a graph corresponds to the motion characteristic of a single pedestrian. This 
figure is in color in the electronic copy of this paper. 





































Figure 7: (a) An output image, (b) the corresponding trajectories of the pedestrians, (c,d) the distance vs 
time graphs, (e,f) the velocity vs time graphs, and (g,h) the acceleration vs time graph of the scenario with 
simultaneous flow of pedestrians in both directions. The trajectory (b) shows separate groups. The red paths 
represent that of the pedestrians from the right and the blue ones represent pedestrians from the left. The 
graphs on the right (d, f, h) emphasizes the intersection of motion characteristics of two groups of pedestrians. 
Graphs of pedestrians from the opposite direction do not start in distance zero to identify separate groups 
in the graph. This figure is in color in the electronic copy of this paper. 
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Figure 8: (a) An output image, (b) the trajectories of the pedestrians, (c) the distance vs time graph, (d) the 
velocity vs time graph, and (e) the acceleration vs time graph of the scenario wherein an individual suddenly 
stops and does a U-turn. This figure is in color in the electronic copy of this paper. 






















Figure 9: (a) An output image, (b) the trajectories of the pedestrians, (c,d) the distance vs time graphs, 
(e,f) the velocity vs time graphs, and (g,h) the acceleration vs time graphs of the counter flow scenario. The 
trajectory (b) shows separate groups. The red paths represent that of the pedestrians from the right and the 
blue ones represent pedestrians from the left. The graphs on the right (d, f, h) emphasizes the intersection 
of motion characteristics of two groups of pedestrians. Graphs of pedestrians from the opposite direction do 
not start in distance zero to identify separate groups in the graph. This figure is in color in the electronic 
copy of this paper. 











































Figure 10: (a) An output image, (b) the trajectory of the pedestrians, (c) the distance vs time graph, (d) the 
velocity vs time graph, and (e) the acceleration vs time graph of the scenario wherein a pedestrian moves 
back and forth, showing variations in his motion characteristics. This figure is in color in the electronic copy 
of this paper. 




































